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(54) Dynamic allocation of buffer space between requests and responses 



(57) The space of a t)uffer 201 is logically parti- 
tioned into space reserved for requests only, space 
reserved for responses only, avd space that can be 
used for either requests or responses, i.a, dynamically 
usable as needed by the system. An axtXter 301 uses 
three registers to keep track of the request buffer space 
302, the response buffer space 303 and the dynamic 
space 304. The artMter compares each of the registers 
with a corresponding limit 307, 308. and 309, to deter- 
mine if a request packet or a response packet should k>e 
sent to the buffer. The limits are set by software and 
define the maximum number of request products, 
response packets, and total number of packets the 
buffer can hold. For example, the limit may be set to 
eight requests, etgfit responses and ten total. Thus two 
spaces are reserved for requests and two spaces are 
reserved for responses, and six are dynamically usable. 
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Description 

TgnHNICAL FIF LD OF THE INVENTION 

This invention relates in general to multHyocessor 
conputer systems arx) in particular to the allocation of 
queue space t>etween requests and responses. 

BACKGROUND QF THE INVENTION 

In prior systems, there are a certain number of 
request buffer locations and a certain numk)er of 
response buffer locations. These buffer locations are 
fixed In the hardware and not changeable bjf software. 
In some systems, the request and response buffer loca- 
tions are on physically separate hardware devices. 
Thus, the fixed nature causes an inherent inefficiency in 
the allocalion of buffer space. Basically, the problem is 
that if a large number of request packets were received 
which fill up the buffer additiomri. request packets woUld 
have to be halted until the request packet buffer is emp- 
tied. This filling and emptying of buffers is tinie consum- 
ing arxi inefficient 

A consequence of this problem is that the buffer 
response time is not constant ttiere is a possitNlity that if 
a n^ packet is one of the type that can be sent out and 
used right away, and if there is no buffer space at that 
time, the packet wil I be delayed until the buffer becomes 
open. This problem greatly affects performance of the 
system. 

It is therefore desirable to have a system that 
makes more efficient use of the available buffer space 
and reduces the probability that a data packet will be 
delayed due to congestion in the queue buffers. 

It is therefore further desirable Id have a system 
that has more efficient distributk>n of data packets and 
positively impacts system performance, by maintaining 
open queue buffer space, and thus allowing better 
throughput of data packets. 

SUMMARY OF THE INVENTION 

The inventk)n partitions a single buffer into space 
reserved for requests only, space reserved for 
responses only, and space that can be used for either 
requests or responses, as needed by the system. Under 
this arrangement, if at any instant one of the reserved 
buffers (requests or responses) goes into overflow, the 
dual use buffers can be used to keep the data flowing 
efficiently. 

The processor agent chip or PAC has three internal 
registers that keep track of the status of the txjffer 
space. One counter keeps track of request buffer space, 
a second counter keeps track of response buffer space, 
and the third counter keeps track of the total amount of 
buffer space that is being used. By using the three 
counters and comparing each of them to ttvee linrtils. 
the system is capable of dynamic buffer allocation. The 



limits are set by software and define the maximum 
number of request packets that can t>e in the txiffer. the 
maximum number of response p»tets tfiat can be in 
the buffer, and the total number of both request and 
5 response packets that can be in the buffer. 

Before the PAC sends a request packet, the PAC 
polls counters to insure that the buffer is not full. The 
PAC checks the request counter to ensure that the 
request limit has not been exceeded, and then the PAC 
10 checks the total counter to ensure this packet will not 
ecceed the total limit count for that buffer. 

For example, assume the limit for the total txjffer 
size is set for ten packets of both response and request 
types, and the limitlortherequeslcount is set for eight 
15 packets of the request type. That wouU mean that the 
request packet could be sent as kmg as there was not 
eight reque^ packets currentiy in the request buffer and 
that there was not cunrenfly ten total packets of both 
request and response types in the buffer. This basically 
20 means that the request packets are limited to a maxi- 
mum of eight of ten kx»tion&. And consequently, the 
other two kx»tions are resen/ed exclusively for the 
response packets. 

Similarly for responses, if the limit for the total buffer 
2S size is set for ten packets of both response and request 
types, and the Bmit for the response count is set for eight 
packets of the response type. That wouM mean that the 
response packet couki be sent as long as there was not 
eight response packets cun^entiy in the response buffer 
30 and ttiat there was not currently ten total packets of txrth 
request and response types in the buffer. This bask^Ily 
means that the response packets are IMted to a maxi- 
mum of eight of ten locations. Consequently, the other 
two locatkHis are reserved exclusively for the request 
36 packets. 

Therefore, with the settings of 8/8/1 0. two k>cations 
are always reserved for requests, two k)cation8 are 
always reserved for responses, and the ren^aining sax 
kxations can be used by for either responses or 
40 requests, as needed by the system. 

A technk»l feature of the invention is to use a single 
buffer for both request buffer space and response buffer 



Another technical feature of the invention is to 
45 resenre some buffer space for only requests, to resen^e 
some buffer space for only responses, and to use the 
remainder of the buff^ space for requests or 
responses, as required by the system. 

A firther technical feature of the invention is to use 
50 counters to keep track of the status of the different types 
of buffer spaces. i.a request, response, and total. 

Afurthertechnk»lfeatureof theinventton istDhave 
the alk>cation of the different types of buff^ spaces soft- 
ware adjustable. 
55 The foregoing has outlined rather broadly the fea- 
tures and technical advantages of the present invention 
in order that the detailed description of the Invention that 
16lk>ws may be better understood. Additional features 
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and advantages of the invention will be descnbed here- 
inafter which forni the sut>ject of the daims of the inven- 
tion. It should be appreciated by those skilled in the art 
that the conception and the specific embocfiment dis- 
closed may be readily utilized as a basis fdr modifying or 
designing other structures for canning out the same 
purposes of the present invention. It should also be real- 
ized by those skilled in the art that such equivalent con- 
structions do not depart from the spirit and scope of the 
invention as set forth in the appended claims. 

BRIEF DESCRIPTION OF THE DRAWINGS 

For a more complete undeislanding of the present 
invention, and the advantages thereof, reference is now 
made to the foltowing descriptions taken in corijunction 
with the accompanying drawings, in whkrfi: 

FIGURE 1 depicts a single node of a multi-node, 
multi-processor system that uses the inventive 
buffer alkx:atk>n; 

FIGURE 2 depicts the arrangement of a single 
cross bar router chip (RAC); 
FIGURE 3 depk;ls the arrangement of a cross bar 
arbiter within the RAC. 

DESCRIPTION OF THE PREFERRED EMBODI- 

FIGURE 1 depicts a single node of a multi-node, 
multi-processor computer system. The overall system 
may have a plurality of these nodes shewn in FIGURE 
1. 

Each node, in the embodiment shown, can support 
up to sixteen processors 110. These processors are 
connected to processor agent chips or PACs 111. The 
functk>nof each PAC 111 is to transmit requests from its 
associated processors 110 through cross bar router 
chips (RAC) 112 to the memory access chips (MAC) 
113 and then forward the responses back to the 
rec^esting processor. Each PAC 111 has an input/but- 
put (I/O) subsystem 117. Each MAC 113 controls 
access to its associated coherent menrtory 114. When 
a. processor 1 10 generates a request to access mem- 
ory (or other resource), the associated PAC 111 sends 
the request through the proper RAC 112 to a MAC 113. 
If the request is destined for menrKsry 114 on the k)cal 
node, MAC 113 accesses the memory attached to it If 
the request is destined for memory on another node, 
MAC1131bnivaidstherequesttoTAC115.TAC 1l5is 
the interface between the node and an SCI ring 116. 
TAC 115 is also known as a torokial access chip or a 
SCI controller. The SCI rings 1 16 (not shewn) intercon- 
nect the nodes in the multi-node system. 

FIGURE 2 depicts a single RAC chip 112 where- 
through packets fbw from PAC 111 to MAC 113, and 
from MAC 1 1 3 to RAC 1 1 1 . RAC chip 1 1 2 forms a cross- 
bar whk:h routes requests and responses in both direc- 



tions. RAC has fbir PAC input buffers 201. one buffer 
associated with a respective PAC. The RAC also has 
four MAC input buffers, one buffer associated with a 
respective MAC. Each of the PAC input buffers 201 can 
5 hoM up to ten packets of both response and request 
types. 

PAC 111 sends a packet to the RAC 112. and 
where it is hekJ in the PAC input buffer 201 until only 
RAC can send the packet on to MAC 113. Similarly for 

10 the opposite direction, then the MAC tries to send a 
packet to a certain PAC. it goes through the proper RAC 
where it first goes into the input buffer on that RAC and 
is later on forwarded on to the proper PAC. The MAC 
input buffers are not part of this inventk>n. The PAC 

IS input buffm will hereinafter be referred to as buffers. 

The processors will make requests for data and 
provide responses for data. For example, there isatime 
when processor 110 will make a request for a certain 
line memory. That request will befbnwarded to PAC 1 1 1. 

so PAC 111 forwards it to one of the two RACs 112 to 
whx:h it is connected, and then to the MAC chip 1 1 3 that 
actually has the requested memory line 1 14. When RAC 
111 sends the request to RAC 112, the request is stored 
in buffer 201. and is later fbnwarded to the MAC 113. If 

25 the request if to a k)cal memory line, MAC will access 
the memory line from memory and send it back as a 
response, to the original PAC. 

There are also times when processor 1 1 0 requests 
a line memory and the line memory is cached in another 

30 processor. In that case, the request goes from PAC 1 1 1 
to MAC 113. and MAC 113 determines that the fine 
memory is at a remote kx»tion. and then sends a 
request to a second PAC to access the line menxxy. 
The second PAC routes the request to the appropriate 

35 processor, arxl then serxte back the data from the mem- 
ory fine as a responsa So in this scenario, the PAC 
sends response across the RAC to the MAC. where It 18 
eventual delivered to the first PAC wherein the request 
originated. 

40 RGURE 3 depkrts the arrangement of aoss bar out 
arbiter 301 that reskies in PAC 111. Arbiter 301. in con- 
junctk>n with buffer 201 perfornrs the dynanuc buffer 
alkx^ation. Each arbiter 301 has three counter registers 
302. 303. 304. Counter 302 is a request counter that 

45 keeps track of the total nurTt>er of requests that are cur- 
rently in use n buffer 201 . Counter 303 is the response 
counter that keeps track of the total number of 
resporses tfiat are currently in use in txjffer 201 . Coun- 
ter 304 is the total counter that keeps track of the tctal 

50 number of buffer locatk>ns cun-ently allocated fa use by 
both responses and requests in buffer 201. Ail three of 
these counters are used as inputs to crossrbar out arbi- 
ter 301. 

The other three inputs to art)iter 301 are the buffer 
55 limits, whfch can, if desired, be set by software. Request 
limit 307 indk»tes the maximum buffer space that can 
be used for request packets. Response limit 308 indi- 
cates the maximum buffer space that can be used for 
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response packets. Limit 309 indcates the total size of 
the txiffer or the total amount of both response and 
request packets that the kxjffer will hokJ. These three 
limits cause registers that can k>e written by the system 
to allow the limits to be flexible and changeable by soft- 5 
ware commands. 

Cross-bar axtxter 301, using data from the limit 
inputs and the counter inputs, decKles whether to send 
a request packet or to send a response packet Sup- 
pose, the PAG has an output directing the dispatch of a 10 
request packet and another output directing the dis- 
patch of a response packet tf the request limit is greater 
than the request counter and the total limit is greater 
than the total counter, then it is safe to send the request 
packet K the response limit is greater than the response is 
counter and the total limit is greater tfian the total coun- 
ter, then it is safe to send the response packet How- 
ever, if one of these groups of concfitk>ns Is not m^, then 
the PAC can send the packet with the conditions that are 
met. thus preventing packet jams. Moreover, when the 20 
total limit is the same or less ttian the total counter, \he 
buffer is full and both packets would be heU until the 
RAC sends informatk>n stating that it has freed up some 
of the txjffer locatk)ns. 

The buffer space reserved fbr request packets is ss 
the total limit minus the response limit for the 8/8/10 
arrangement discussed prevtousiy. This arrangement 
yields a reserved request space of two. The t)uffier 
space reserved for response packets is the total limit 
minus the request limit for the 8/S/10anfangenientdis- ^ 
cussed pr^iously, yielding a reserved response space 
of twa The remainder of the buffer space, called the 
dynamic space is equal to the response limit plus the 
request limit minus the total limit, for the 8/8/10 arrange- 
ment discussed prGvk)usly and yiehls a dynamic space 3S 
of six. 

Arrangements other than 8/8/1 0 are possible, how- 
ever, at least one space shouki always be reserved fbr 
response packet& OthenMse. if the buffer were to be 
filled up wHh requests, no response packets could be 40 
moved, and the system could be deed-k)ckad. This 
would yieM a 9/10/10 anangement i a a maxinrum of 
nine requests, a maximum of ten responses, of a ten 
packet buffer. The 9/10/10 system is the most dynanuc 
system, with a dynamic space of nine. The least 45 
dynarnc system wouU be a 5/5/10. with a dynanruc 
space of zera this effectivety is the prior art arrange- 
ment. 

The 8/8/10 arrangement means at least a total of 
four spaces are reserved, andi six spaces are dynanuc. so 
This anangement has been found to woric well at keep- 
ing the packets moving. A 7/7/10 is less flexible, with 
only four k)cations that are dynamically alkx:able, 
t)ecause three spaces are reserved for requests and 
three more for responses. 55 

Whenever the RAC sends a request packet it incre- 
ments both the request counter and the total counter. 
The RAC stores the request until the MAC is ready to 



receive it When the request packet is sent on to the 
MAC. then the RAC sends a count of the actual number 
of requests that have been sent out to the MACs. When 
the PAC receives this count infbrmatk)n from the RAC, 
the PAC subtracts the number of sent out packets from 
both the request counter and the total counter. Thus, the 
counters are decremented when the RAC sends out the 
request. 

Whenever the PAC sends a response packet it 
increments both the response counter and the total 
counter. The RAC stores the response until the MAC is 
ready to receive It. When the response packet is sent on 
to the MAC, then the RAC sends a count of the actual 
number of responses that have been sent out to the 
MACS. When the PAC receives this count ink)rmalk)n 
from the R/VC. the PAC subtracts the number of sent out 
packets from both the response counter and the total 
counter. Thus, the counters are deaemented wtien the 
RAC sends out the response 

The RAC has no knowledge of the actual alkxatton 
of the buffer space between request and response. The 
RAC knows that it is receiving requests and responses 
packets, and the numt)er of packets of each type being 
sent out. The RAC sends this information back to the 
PAC. Buffer 201 is kieally physk^Dy one buffer and is 
broken up k>gk:ally into request buffers and responses 
buffers. However, the system coukJ work with multiple 
physical registers. The RAC knows which buffer kx;a- 
tk)ns are whk^h, and whk;h buffer locatk>ns currently 
contain responses, but it does not receive infonratkxi 
about the counters and Imnits in the RAC. 

Although the present inventkHi and Hs advantages 
have t>een desaibed in detail, it shouM t>e understood 
ttiat various cfianges, sut)stitutions andi alteratkxis can 
be made herein without departing from the spirit arxi 
scope of the invention as defined by the appended 
claims. 

Claims 

1. A computer system having a processor that creates 
request packets to access memory and creates 
response packets that satisfy queries made by the 
system, the system comprising: 

a dynanvcally allocable buffer space for storing 
and routing the request packets and the 
response packets; and 

an arbiter for receiving the request packets and 
the response packets from the processor, and 
for making determinations whether to send the 
request packets and the response packets to 
the buffer space . 

2. The computer system of claim 1, wherein said 
determinations are made in part, under control of: 

a request counter that stores a number value 
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representing a total number of request packets 
cun^ently stored in ttie Ixjffer space: 
a response counter that stores a number value 
representing a total number of response pack- 
els currently stored In tfie buffer space; 5 
a total counter that stores a number value rep- 
resenting a total number of both request pack- 
ets and response packets currently stored in 
the buffer space: 

a request limtt value representing a masdmum io 
number of request packets that can be stored 
in the txjffer space; 

a response limit value representing a maximum 
number of response pactols that can be stored 
in the buffer space; and is 
a total limit value representing a maximum 
number of both request packets and response 
packets that can be stored In the buffer spaca 

3. The computer system of daim 2, wherein: 20 

the artxter compares the request limit with the 
request counter and compares the total limit 
with the total counter to detennine whether to 
send the request packet to the buffer space; 2S 
If the request limit is greater than the request 
counter, and if the total limit is greater than ttie 
total counter, then the art>iter will send tfie 
request packet and increment kx>th the request 
counter and the total counter; 30 
if the request limit is not greater than the 
request counter, then the astiter compares the 
response limit with the response counter and 
conpares the total limit with the total counter to 
detemrtine whether to send the response as 
packet to the buffer space; and 
rf the total fimit is not greater than the total 
counter, then the arbiter will send neither the 
request packet nor the response packet until 
the total limit is greater than the total counter. 40 

4. The computer system as in daim 2 or 3. wherein: 

the buffer space transmits the request packet 
to a locatnn in the system after receiving the 45 
request packet from the arbiter, and then relays 
transmissk)n infonratnn to the artxter; and 
the arbiter receives the transmissk)n informa- 
tion and decrements the request counter and 
the total counter. so 

5. The corrputer system of daim 2. wherein: 

the art)iter compares tfie response limit with the 
response counter and compares the total limit ss 
with the total counter to determine whether to 
send the response packet to the buffer space: 
if the response limit is greater than the 



response counter, and if the total limit is greater 
than the total counter, then the aibiter wfll send 
the response packet and irK:rement kxHh the 
response counter and the total counter; 
If the response limit is not grater than the 
response counter, then the arbiter conrpares 
the request limit with the request counter and 
compares the total limit with the total counter to 
determine whether to send the request packet 
to the buffer space; and 
if the total limit is not greater than the total 
counter, then the ait)iter will send neither the 
request packet nor the response packet until 
the total limit Is greater than the total counter. 

6. The computer system as tfi daim 2 or 5. wherein: 

the buffer space transmits the response packet 
to a location in the system after receiving the 
response packet from the art)iter. and then 
relays transmisskm lnft>rmatk)n to the arbiter; 
and 

the arbiter receives the transmission informa- 
tk>n and decrements the response counter and 
the total counter. 

7. The computer system as in one of claims 2-6. 
wherein: 

the request limit, the response limit, and the 
total limit are alterable by software commands. 

8. The computer system as in one of claims 2-7. 
wherein: 

a reserved request space of the buffer space Is 
equal to the total limit nrwius the response llrnit; 
a reserved response space of the buffer space 
is equal to the total limit minus the request limit; 
and 

a dynamnally allocable space of the buffer 
space is equal to the request Bmit plus the 
response limit minus the total KmH. 

9. The computer system as in one of dalms 2-8, 
wfierein: 

the request limit Is 8; 

the response limit is 8; 

the total limit is 10; 

the reserved request space is 2; 

the reserved response space is 2; and 

the dynamtoally anocable space is 6. 

ia A method for dynamically alkxating buffer space in 
a corrputer system, the method comprising the 
steps d: 
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accepting request packets to access a mem- 
ory: 

accepting response packets that satisfy cpie- 
ries made the system; 
detenmining whether to send the request pack- 
ets and the response packets to the txrffer 
space; 

dynamically allocating the buffer space so as to 
accommodate a maximum number of both 
request packets and response packets: and 
storing the request packets and the response 
packets in the buffer space. 
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